(N 

o 



o 

H 
c/3 



- 1- < 

>< 



Consistency of the posterior distribution and MLE for 
piecewise linear regression 

Tristan Launay 1,2 Anne Philippe 1 Sophie Lamarche 2 

1 Laboratoire de Mathematiques Jean Leray, 
2 Rue de la Houssiniere - BP 92208, 44322 Nantes Cedex 3, France 



Electricite de France R&D, 1 Avenue du General de Gaulle 
92141 Clamart Cedex, France 



Abstract 



We prove the weak consistency of the posterior distribution and that of the Bayes estimator 
for a two-phase piecewise linear regression mdoel where the break-point is unknown. The 

i-G non-differentiability of the likelihood of the model with regard to the break-point parameter 

induces technical difficulties that we overcome by creating a regularised version of the problem 
at hand. We first recover the strong consistency of the quantities of interest for the regularised 
version, using results about the MLE, and we then prove that the regularised version and the 
original version of the problem share the same asymptotic properties. 
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We consider a continuous segmented regression model with 2 phases, one of them (the rightmost) 
being zero. Let u be the unknown breakpoint and 7 £ M. be the unknown regression coefficient of 
the non zero phase. The observations X\-_ n — (X\, . . . , X n ) depend on an exogenous variable that 
we denote t\- n = (ti, . . . , t n ) via the model given for i — 1, . . . , n by 



Xi = n(ri,U) + & ■= 7 ' (U - u)lfo, +«,[(«) + 6. (1-1) 



Vh ■ 

where (^j)ieN is a sequence of independent and identically distributed (i.i.d.) random variables 

with a common centered Gaussian distribution of unknown variance a 2 , AT(0,<t 2 ), and where 1^ 

denotes the indicator function of a set A. 

Such a model is for instance used in practice to estimate and predict the heating part of the 
electricity demand in France. See Bruhns et al. (2005) for the definition of the complete model 
and Launay et al. (2012) for a Baycsian approach. In this particular case, u corresponds to the 
heating threshold above which the temperatures t\ :n do not have any effect over the electricity 
load, and 7 corresponds to the heating gradient i.e. the strength of the described heating effect. 

The work presented in this paper is most notably inspired by the results developed in Ghosh ct al. 
(2006) and Feder (1975). 

Feder proved the weak consistency of the least squares estimator in segmented regression 
problems with a known finite number of phases under the hypotheses of his Theorem 3.10 and 
some additional assumptions disseminated throughout his paper, amongst which we find that the 
empirical cumulative distribution functions of the temperatures at the n-th step t n i , ■ ■ ■ , t nn are 
required to converge to a cumulative distribution function, say F n converges to F , which is of course 
to be compared to our own Assumption (Al). Feder also derived the asymptotic distribution of 
the least squares estimator under the same set of assumptions. Unfortunately there are a few 
typographical errors in his paper (most notably resulting in the disappearance of a 2 from the 



asymptotic variance matrix in his main theorems), and he also did not include ct 2 in his study of 
the asymptotic distribution. 

The asymptotic behaviour of the posterior distribution is a central question that has already 
been raised in the past. For example, Ghosh et al. worked out the limit of the posterior dis- 
tribution in a general and regular enough i.i.d. setup. In particular they manage to derive the 
asymptotic normality of the posterior distribution under third-order differentiability conditions. 
There are also a number of works dealing with some kind of non regularity, like these of Sareen 
(2003) which consider data the support of which depends on the parameters to be estimated, or 
those of Ibragimov and Has'minskii (1981) which offer the limiting behaviour of the likelihood 
ratio for a wide range of i.i.d. models whose likelihood may present different types of singular- 
ity. Unfortunately, the heating part model presented here does not fall into any of these already 
studied categories. 

In this paper, we show that the results of Ghosh ct al. can be extended to a non i.i.d. two- 
phase regression model. We do so by using the original idea found in Sylwester (1965) 1 : we 
introduce a new, regularised version of the problem called pseudo-problem, later reprised by Fcdcr. 
The pseudo-problem consists in removing a fraction of the observations in the neighbourhood of 
the true parameter to obtain a differentiate likelihood function. We first recover the results 
of Ghosh et al. for this pseudo-problem and then extend these results to the (full) problem by 
showing that the estimates for the problem and the pseudo-problem have the same asymptotic 
behaviour. 

From this point on, we shall denote the parameters 9 = (7, u, ct 2 ) = (7?, ct 2 ) and 9q will denote 
the true value of 9. We may also occasionally refer to the intercept of the model as (3 = —ju. The 
log-likelihood of the n first observations X 1: „ of the model will be denoted 

n 

lun(X Un \0) ='jr i li(X i \9) (1.2) 

= -2 bg (2™ 2 ) - J2 5^2 (** " 7 • (*< - ")%, +oo[K) 2 , (1-3) 

i— 1 

where li(Xi\9) designates the log-likelihood of the i-th observation X^ i.e. 

h(X 1:n \e) = ~ log (2^ct 2 ) - -L (X 4 - 7 • (U - u)t [u , +oo[ (u)) 2 . (1.4) 

Notice that we do not mention explicitly the link between the likelihood I and the sequence of 
temperatures (i ra ) n eN in these notations, so as to keep them as minimal as possible. The least 
square estimator 9 n of 9 being also the maximum likelihood estimator of the model, we refer to it 
as the MLE. 

Throughout the rest of this paper we work under the following assumptions 

Assumption (Al). The sequence of temperatures (exogenous variable) (i n )neN belongs to a 
compact set [u, u] and the sequence of the empirical cumulative distribution functions (F n ) n &N of 
(ti,...,t n ), defined by 

1 " 

F n (u) = -Vl[(,. +00 [(«), 

n * — ' 

i=\ 

converges pointwise to a function F where F is a cumulative distribution function itself, which is 
continuously differentiable over \u, u]. 

Remark 1. Due to a counterpart to Dini's Theorem (see Theorem 7.1 taken from Polya and Szcgo, 
2004, (p81)), F n converges to F uniformly over [u, u). 



1 Sylwester indeed considers the same model as we do here, however his asymptotic results are false due to an 
incorrect rcparamctrisation of the problem and an error in the proof of his Theorem 3.5. 



Remark 2. Let h be a continuous, bounded function on \u, u]. As an immediate consequence of 
this assumption, for any interval / C [u, u] , we have, asn-) +00 



^Y^h{U)\i{ti) = f h(t)dF n (t) -> j h(t)dF{t) = J 



h(t)f(t) dt, 



the convergence holding true by definition of the convergence of probability measures (see Billingsley, 
1999, pages 14-16). In particular, for / = \u, u] and I =] — 00, u] we get, as n —} +00 

-YsKU)^ h(t)f(t)dt, -J2h(U)l [tit+oo[ (u)^ h(t)f(t)dt. 

n i=i J a n i=i J a- 

Remark 3. It is a general enough assumption which encompasses both the common cases of 
i.i.d. continuous random variables and periodic (non random) variables under a continous (e.g. 
Gaussian) noise. 

Assumption (A2). 9q e 0, where the parameter space is defined (for identifiability) as 

= R*x]u 1 u[xR* + , 

where R* = {x £ K , x ^ 0} and W + = {x E R , x > 0}. 

Assumption (A3). / = F' does not vanish (i.e. is positive) on }u, u[. 

Assumption ( A4) . There exists K C O a compact subset of the parameter space such that 

9„ <G K for any n large enough. 

The paper is organised as follows. In Section 2, we present the Bayesian consistency (the 
proofs involved there rely on the asymptotic distribution of the MLE) and introduce the concept 
of pseudo-problem. In Section 3, we prove that the MLE for the full problem is strongly consistent. 
In Section 4 we derive the asymptotic distribution of the MLE using the results of Section 3: to do 
so, we first derive the asymptotic distribution of the MLE for the pseudo-problem and then show 
that the MLEs for the pseudo-problem and the problem share the same asymptotic distribution. 
We discuss these results in Section 5. The extensive proofs of the main results are found in Section 
6 while the most technical results are pushed back into Section 7 at the end of this paper. 
Notations. Whenever mentioned, the O and o notations will be used to designate a.s. O and a.s. 
o respectively, unless there are indexed with P as in Op and op, in which case they will designate 
O and o in probability respectively. 

Hereafter we will use the notation A c for the complement of the set A and B(x, r) for the open 
ball of radius r centred at x i.e. B(x, r) = {x' , j|ir' — x\\ < r}. 

2 Bayesian consistency 

In this Section, we show that the posterior distribution of 9 given (Xi, . . . ,X n ) asymptotically 
favours any neighbourhood of #0 as long as the prior distribution itself charges a (possibly different) 
neighbourhood of 9q (see Theorem 2.1). We then present in Theorem 2.2 the main result of this 
paper i.e. the convergence of posterior distribution with suitable normalisation to a Gaussian 
distribution. 

2.1 Consistency and asymptotic normality of the posterior distribution 

Theorem 2.1. Let tt(-) be a prior distribution on 9, continuous and positive on a neighbourhood 
of 9q and let U be a neighbourhood of 9q, then under Assumptions (A1)-(A4), as n — > +00, 



a.s. 



n(9\X 1:n )dO^>l. (2.1) 

u 



Proof for Theorem 2.1. The proof is very similar to the one given in Ghosh and Ramamoorthi 
(2003) for a model with i.i.d. observations. Let S > small enough so that B(9q,S) C U. Since 



/ ir(9\X 1:n )d9 = 
Ju 



1 



€ 



J uc ir(0)c!x P [i 1:n (x 1:n \e)-i 1:n (x 1:n \0 o )]de 

J v tt(9) exp[l 1:n (X 1:n \9) - li :n (X 1:n \e )} d9 

1 
/ B c (eo , s) T(0)exp[Z 1: „(X 1:n |0)-Z 1:n (X 1: „|0 o )]d0 
1 + Ib(8 ,5) <°) cMh:n(X 1:n \9) ~ h.. n (X 1:n \0 o )] d9 

it will suffice to show that 

/flc(tfo,«) ^Wex P [/i:„(X 1:n |0) - h :n (X Un \6 )] d9 0i _ 



/s(eo,«5) ""^ exp[/ 1: „(X 1:n |6>) - ii : „(JCi: n |flo)] 



d0 



0. 



(2.2) 



To prove (2.2) we adequately majoratc its numerator and minorate its denominator. The 
majoration mainly relies on Proposition 7.11 while the minoration is derived without any major 
difficulties. The comprehensive proof of (2.2) can be found in Section 6.1 on page 11. □ 

Let 9 € 0, we now define 1(9), the asymptotic Fisher Information matrix 1(9) of the model, 
as the symmetric matrix given by 



1(9) = 



(t-ufdF(t) -cr" 2 7 / (t-u)dF(t) 



1 dF(t) 



-2 2 

a 7 



r 



(2.3) 



It is obviously positive and definite since all its principal minor determinants are positive. The 
proof of the fact that it is indeed the limiting matrix of the Fisher Information matrix of the model 
is deferred to Lemma 7.10. 

Theorem 2.2. Let ir(-) be a prior distribution on 9, continuous and positive at 9q, and let fco £ N 
such that 



||0|| fco ?r(0)d0<+oo, 



id denote 



t = n*(9-e n ), 



(2.4) 



and TT n (-\Xi :n ) the posterior density oft given X\. n , then under Assumptions (Al)-(A^), for any 
$J k $J fco, as n —> +oo, 



\t\ 



n n (t\X 1:n )^(27r)-i\I(9 )\ie-i t ' I ^ t 



dt^O, 



(2.5) 



where 1(9) is defined in (2.3) and 9q the true value of the parameter. 



The proof Theorem 2.2 relies on the consistency of the pseudo-problem, first introduced in 
Sylwester (1965), that we define in the next few paragraphs. 



2.2 Pseudo-problem 

The major challenge in proving Theorem 2.2 is that the typical arguments usually used to derive 
the asymptotic behaviour of the posterior distribution (see Ghosh et al., 2006, for example) do 
not directly apply here. The proof provided by Ghosh et al. requires a Taylor expansion of the 
likelihood of the model up to the third order at the MLE, and the likelihood of the model we 
consider here at the n-th step is very obviously not continuously diffcrcntiablc w.r.t. u in each 
observed temperature ij, i = 1, . . . , n. Note that the problem only grows worse as the number of 
observations increases. 

To overcome this difficulty we follow the original idea first introduced in Sylwcster (1965), and 
later used again in Fcdcr (1975): we introduce a pseudo-problem for which we are able to recover 
the classical results and show that the differences between the estimates for the problem and the 
pseudo-problem are, in a sense, negligeable. The pseudo-problem is obtained by deleting all the 
observations within intervals D n of respective sizes d n centred around uq. The intervals D n are 
defined as 



D n = 



uq - —, u a + — 



and their sizes d n are chosen such that asn-> +oo 

d n ->• 0, n"5(lo gn ) • d' 1 -+ 0. (2.6) 

This new problem is called pseudo-problem because the value of uq is unknown and we therefore 
cannot in practice delete these observations. Note that the actual choice of the sequence (d n )neN 
does not influence the rest of the results in any way, as long as it satisfies to conditions (2.6). It 
thus does not matter at all whether one chooses (for instance) d„ = n~i or d n = log~ n. 

Let us denote n** the number of observations deleted from the original problem, and n* = 
n — n** the sample size of the pseudo-problem. Generally speaking, quantities annotated with a 
single asterisk * will refer to the pseudo-problem. l\. n {Xi- n \9) will thus designate the likelihood 
of the pseudo-problem i.e. (reindexing observations whenever necessary) 

* «* -. 

l*l:JXl:n\0) = ~ tog (2^ 2 ) - £ — (X* - 7 • (U - «)1 [U , +oo[ (u)) ' . (2.7) 

1=1 

On one hand, from an asymptotic point of view, the removal of those n** observations should 
not have any kind of impact on the distribution theory. The intuitive idea is that deleting n** 
observations takes away only a fraction n** /n of the information which asymptotically approaches 
zero as will be shown below. The first condition (2.6) seems only a natural requirement if we ever 
hope to prove that the MLE for the problem and the pseudo-problem behave asymptotically in a 
similar manner (we will show they do in Theorem 4.2, see equation (4.1)). 

On the other hand, assuming the MLE is consistent (we will show it is, in Theorem 3.3) 
and assuming that the sizes d n are carefully chosen so that the sequence (u n )neN falls into the 
designed sequence of intervals (£>„)„<=« (see Proposition 4.1, whose proof the second condition 
(2.6) is tailored for), these regions will provide open neighbourhoods of the MLE over which the 
likelihood of the pseudo-problem will be differentiable. The pseudo-problem can therefore be 
thought of as a locally regularised version of the problem (locally because we are only interested 
in the differentiability of the likelihood over a neighbourhood of the MLE). We should thus be 
able to retrieve the usual results for the pseudo-problem with a bit of work. It will be shown that 
this is indeed the case (see Theorem 2.3). 

If the sequence (c?„)„gN satisfies to conditions (2.6), then as n — » +oo, 

n** n* 

n n 



Using the uniform convergence of F n to F over any compact subset (see Assumption (Al), and 
its Remark 1), we indeed find via a Taylor-Lagrange approximation 

= f(u + ^-f(u -^+o(1) 

= d„ ■ f(u n ) + o(l), 

where u n € D ni so that in the end, since u n — > uq and / is continuous and positive at uq, we have 
a.s. 

71** 

= d n ■ (/(«o) + o(l)) + o(l) -> 0. 

n 

We now recover the asymptotic normality of the posterior distribution for the pseudo problem. 

Theorem 2.3. Let tt(-) be a prior distribution on 9, continuous and positive at 9q, and let kg € N 
such that 

\\6\\ ko Tr(6)d6 < +00. 
e 

and denote 

t* = nHe-e* n ), (2.8) 

and 7r*(-|Xi : „) the posterior density oft* given X\- n , then under Assumptions (Al)-(A4) and 
conditions (2.6), for any ^ k ^ ko, as n — > +00, 



\t\\ k 



r n (t\X lM ) - (27r)-i |/(0 o )|^e-^' 7 ^* d* ^^ 0, (2.9) 



where 1(9) is defined in (2.3). 

Proof of Theorem 2.3. The extensive proof, to be found in Section 6.1, was inspired by that of 
Theorem 4.2 in Ghosh ct al. (2006) which deals with the case where the observations X\, . . . , X n 
are independent and identically distributed and where the (univariate) log-likelihood is differen- 
tiablc in a fixed small neighbourhood of 9q. Wc tweaked the original proof of Ghosh et al. so that 
we could deal with independent but not identically distributed observations and a (multivariate) 
log- likelihood that is guaranteed differentiable only on a decreasing small neighbourhood of 9$. □ 

2.3 From the pseudo-problem to the original problem 

We now give a short proof of Theorem 2.2. As we previously announced, it relies upon its coun- 
terpart for the pseudo-problem, i.e. Theorem 2.3. 

Proof of Theorem 2.2. Recalling the definition of t and t* given in (2.4) and (2.8) we observe that 

t = t* + ni(dl-e n ). 

Thus the posterior distribution of t* and that of t, given X\._ n are linked together via 

7Tn(t|*l:n) = 7T*(* - a n \X 1:n ) (2.10) 

where 

a n = n^(9* n -9 n ). 



Relationship (2.10) allows us to write 

/ \\t\\ k r n (t-a n \X 1:n )-(2Tr)-i\I(e )\h-i t ' I ^ t 



at 



df 



\\t + a n \\ k n* n (t\X 1:n ) - (27r)-l\I(e )\h-^ t+a -y i ^^+ a ^ 



dt 



< / \\t + a n \\ k n; i (t\X 1:n )-(2n)-i\I(e Q )\h-^' I ^ t 



dt 



+ {2it)-i\I{9oW 



\t + Ur, 



e -i(t+a n yi(8 a )(t+a n ) _ e ~\t' 'l(6 )t 



dt 



Theorem 2.3 ensures that the first integral on the right hand side of this last inequality goes to 
zero in probability. It therefore suffices to show that the second integral goes to zero in probability 
to end the proof, i.e. that asn-> +00 



\t + a r , 



e 2 



i(t+a„)'/(0o)(*+a„) _ p -^t'l(e )t 



dt^>0. 



(2.11) 



But the proof of (2.11) is straightforward knowing that a n — > (see (4.1)) and using dominated 
convergence. □ 

As an immediate consequence of Theorem 2.2 we want to mention the weak consistency of the 
Bayes estimator. 

Corollary 2.4. Let tt(-) a prior distribution on 9, continuous and positive at 0$, such that 

f ||0||7r(0)d0<+oc, 
Je 



and denote 



0ir n (e\X 1:n )d0, 



the Bayes estimator of 9 in the problem. Then under Assumptions (Al)-(AJ^), as n —> +00, 

n*(0„-0„)4o. 

Proof of Corollary 2.4- By definition, 



0n= / 9Tr n (9\X 1:n )d9 
Je 



and this allows us to write 



n»(«»-«n)= / n*($-6 n )ir n (0\X lin )dO 



I tn n {t\X 1:n ) dt 4 0, 

Jk 3 



the last convergence being a direct consequence of Theorem 2.2 with fco = 1. 



□ 



Observe that, under conditions (2.6), the same arguments naturally apply to the pseudo- 
problem and lead to a strong consistency (a.s. convergence) of its associated Bayes estimator due 
to Theorem 2.3, thus recovering the results of Ghosh et al. (2006) for the regularised version of 
the problem. 



3 Strong consistency of the MLE 

In this Section we prove the strong consistency of the MLE over any compact set including the 
true parameter (see Theorem 3.1). It is a prerequisite for a more accurate version of the strong 
consistency (see Theorem 3.3) which lies at the heart of the proof of Theorem 2.3. 

Theorem 3.1. Under Assumptions (Al)-(A^), we have a.s., as n — > +oo, 

||0„-0 o ||=o(l). 

Proof of Theorem 3.1. Recall that K is a compact subset of 9, such that 6 n € K for any n large 
enough. We denote 

h:n(Xu n \S) = sup lu n (Xi :n \6) , for any S C K, 

ess 

K n (a) = {6eQ, W: n {X lM \6) > log a + h-. n {X 1 ., l \K)} , for any a e]0, 1[. 
All we need to prove is that 

3ae]0, 1[, P| lim sup ||0 - 6 \\ = ) = 1. (3.1) 

\ n ^ +oa eeK„(a) J 

since for any n large enough we have 8 n £ K n (a) for any a g]0, 1[. We control the likelihood 
upon the complement of a small ball in K and prove the contrapositive of (3.1) using compacity 
arguments. The extensive proof of (3.1) is to be found in Section 6.2 . □ 

We strengthen the result of Theorem 3.1 by giving a rate of convergence for the MLE (see 
Theorem 3.3). This requires a rate of convergence for the image of the MLE through the regression 
function of the model, that we give in the Proposition 3.2 below. 

Proposition 3.2. Under Assumptions (Al)-(A4), as n — > +oo, a.s., for any open interval I C 
[u, u], 



min \fi(rj n ,U) - fj,(r] ,ti)\ = O (n 2 logn) . 



uei, 

Proof of Proposition 3.2. The proof is given in Section 6.2. □ 

Theorem 3.3. Under Assumptions (Al)-(A4), we have a.s., as n — > +00, 



70 



= 0(71-3 logn). (3.2) 



Proof of Theorem 3.3. We show that a.s. (3.2) holds for each coordinate of 9 n — 6q. The calcu- 
lations for the variance a 2 are pushed back into Section 6.2. We now prove the result for the 
parameters 7 and u. It is more convenient to use a reparametrisation of the model in terms of 
slope 7 and intercept /3 where j3 = — yu. 

Slope 7 and intercept /3. Let V\ and V% be two non empty open intervals of ]u, uq[ such that 
their closures V\ and V2 do not overlap. For any (^1,^2) € V\ X V2, define M(t\,t2) the obviously 
invertible matrix 



M(h,t 2 ) 
and observe that for any r = (/3, 7), 

M(ti,t 2 )T = 



1 h 
1 ti 






Observe that by some basic linear algebra tricks we are able to write for any (ii,i2) 6 Vi X Vj 
||r„-ro||cx» = ||M(ti,t2) _1 M(t l! t2)(r n -ro)||o 

<|||M(ti l t2)~ 1 ||| o-||M(ti,t2)f„-M(tl ) t 2 )'n)||oo 

|«a| + |ti| + 2 



€ 



1*2 - h 



\M(t 1 ,t 2 )? n -M(t 1 ,t 2 )T \\ c 



Thus, using the equivalence of norms and a simple domination of the first term of the product in the 
inequality above, we find that there exists a constant C £ R+, such that for any (ti, £2) €E V\ x V2 

\\Tn-T \\ <C.||M(ti > t 2 )f„-M(ti > t2)7l)||, 



rolUC 



^{n{rjn,U) - ^{m-,U)t 



(3.3) 



Taking advantage of Proposition 3.2, we are able to exhibit two sequences of points (ti, n )neN m 
Vi and (£2.n)nGN m V2 such that a.s., for i = 1, 2 

|M(?n,*i,re) - M(%)*i,n)l = ° (n~ 5 log 71 J . (3.4) 

Combining (3.3) and (3.4) together (using ti = ti, n for every n), it is now trivial to see that a.s. 

1 1 TV. - T oll = O (n** lognj , 

which immediately implies the result for the 7 and f3 components of 9. 

Break-point u. Recalling that u — — /37" 1 and thanks to the result we just proved, wc find 

that a.s. 



-Mn 1 = - 



(3 + O ( ?i 2 log n ) 70 + O ( n 2 log ti 



-A)7o J + O (n 2 lognj = 7i + O fn 2 log/ij 



D 



4 Asymptotic distribution of the MLE 

In this Section we derive the asymptotic distribution of the MLE for the pseudo-problem (see 
Proposition 4.1) and then show that the MLE of pseudo-problem and that of the problem share 
the same asymptotic distribution (see Theorem 4.2). 

Proposition 4.1. Under Assumptions (Al)-(A^) and conditions (2.6), as n —> +cxj 



* n -$ j AA^O,/^)- 1 ), 

where the asymptotic Fisher Information Matrix /(•) is defined in (2.3). 

Proof of Theorem 4-1- The proof is divided in two steps. We first show that the likelihood of the 
pseudo-problem is a.s. differentiable in a neighbourhood of the MLE (9* for TV large enough. We 
then recover the asymptotic distribution of the MLE following the usual scheme of proof, with a 
Taylor expansion of the likelihood of the pseudo-problem around the true parameter. The details 
of these two steps are given in Section 6.3. □ 



Theorem 4.2. Under Assumptions (Al)-(A^) and conditions (2.6), as n — > +00, 

where the asymptotic Fisher Information Matrix /(■) is defined in (2.3). 

Proof of Theorem 4-2. It is a direct consequence of Proposition 4.1 as soon as we show that as 
n — > +00 



6 n -d* n = op (ns). (4.1) 

To prove (4.1), we study each coordinate separately. For 7 and u, we apply Lemmas 4.12 and 4.16 
found in Fcdcr (1975) with a slight modification: the rate of convergence d n he uses may differ 
from ours but it suffices to formally replace (log log n) 2 by (\ogn) all throughout his paper and 
the proofs he provides go through without any other change. We thus get 



% - In = op ( n 2 J , u n - u* n = o P In 2 J . (4.2) 

It now remains to show that 

d 2 n -d 2 n *=o r (n-i). (4.3) 

To do so, we use (4.2) and the decomposition (6.40) 

n „ n 1 n 

°l = ~Y. ^«) + - 12 "«(»&>)& + -12£> 

n *— ' n *—' n ^^ 

i— 1 z— 1 i— 1 

where Vi{rj n ) = 7o' (^~ u o)l[t ij + 00 [(tto)^7)i' (£i — w n )l[ ti]+00 [(u n )- The details of this are available 
in Section 6.3. D 

5 Discussion 

In this Section, we summarise the results presented in this paper. The consistency of the posterior 
distribution for a piccewise linear regression model is derived as well as its asymptotic normality 
with suitable normalisation. The proofs of these convergence results rely on the convergence of 
the MLE which is also proved here. In order to obtain all the asymptotic results, a regularised 
version of the problem at hand, called pseudo-problem, is first studied and the difference between 
this pseudo-problem and the (full) problem is then shown to be asymptotically ncgligeable. 

The trick of deleting observations in a diminishing neighbourhood of the true parameter, orig- 
inally found in Sylwester (1965) allows the likelihood of the pseudo-problem to be differentiated 
at the MLE, once the MLE is shown to asymptotically belong to that neighbourhood (this re- 
quires at least a small control of the rate of convergence of the MLE). This is the key argument 
needed to derive the asymptotic distribution of the MLE through the usual Taylor expansion of 
the likelihood at the MLE. Extending the results of Ghosh et al. (2006) to a non i.i.d. setup, the 
asymptotic normality of the posterior distribution for the pseudo-problem is then recovered from 
that of the MLE, and passes on almost naturally to the (full) problem. 

The asymptotic normality of the MLE and the posterior distribution are proved in this paper 
in a non i.i.d. setup with a non continuously diffcrentiablc likelihood. In both cases we obtain 
the same asymptotic results as for an i.i.d. regular model: the rate of convergence is %Jn and 
the limiting distribution is Gaussian (see Ghosh et al., 2006; Lehmann, 2004). For the piece- 
wise linear regression model, the exogenous variable t 1: „ does not appear in the expression of 
the rate of convergence as opposed to what is known for the usual linear regression model (see 
Lehmann, 2004): this is due to our own Assumption (Al) which implies that t[. n ti- n is equiv- 
alent to n. Note that for a simple linear regression model, we also obtain the rate y/n under 
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Assumption (Al). In the litterature, several papers already highlighted the fact that the rate of 
convergence and the limiting distribution (when it exists) may be different for non regular models 
in the sense that the likelihood is cither non continuous, or non continuously diffcrcntiablc, or 
admits singularities (see Dacunha-Castelle, 1978; Ghosh et al., 1994; Ghosal and Samanta, 1995; 
Ibragimov and Has'minskii, 1981). For the picccwisc regression model, the likelihood is continuous 
but non continuously diffcrcntiablc on a countable set (but the left and right derivatives exist and 
are finite) : the rate of convergence \fn is not so surprising in our case, because this rate was al- 
ready obtained for a univariate i.i.d. model the likelihood of has the same non regularity at a single 
point. In that case, the rate of convergence of the MLE is shown to be n (see Dacunha-Castelle, 
1978, for instance). 

6 Extensive proofs 

6.1 Proofs of Section 2 

Proof of Theorem 2.1. To prove (2.2), we proceed as announced and deal with numerator and 
denominator in turn. 

Majoration. From Proposition 7.11 with p n = 1, for any given e > 0, we can choose S > 
small enough so that a.s. for any n large enough 

Sup -[h:n{X 1:n \6) - Ji : „(Xl:»|0o)] < "C- 

6»eB c (6» ,i5) n 

We thus obtain a.s. for any n large enough 

0< / 7r{e)exp[l ltn (X lm \e)-h in (X 1 . n \0 o )]60 

Jb°(0 ,8) 

^e-" e f n(9)d9. (6.1) 

JB<=(6 ,5) 



Minoration. Define 9 n € B(9 ,S) such that 

inf —[h:n(Xi :n \9) — h:n(Xi; n \9o)] = — [h:n(Xi :rl \9 n ) — h:n(Xl. n \do)] 
QOzB^OqjS) 71 ft 



It is possible to define such a 9 n because B(0q, S) is a compact subset of for 5 > small enough 
and li;n(Xi :n \') is continuous as a function of 0. Let now 

/ 2 2 \ II 71 

b n {9) =^_l-log^+-.-]r Mr*, U) - M, U)} 2 . (6.2) 

v ' i—l 

Recalling the definition of the log-likehood given in (1.2) and replacing Xi by its expression given 
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in (1.1) we find via straightforward algebra 

-[h:n{X 1 .. n \6) - MX l!B |0o)] = l0 S 4 + ( -2 A) ( - E^ J 

n a 2 \a 2 a£J \n*-[ J 

1 - .21™ 



io g i 



i i 



E^-^o + ^o 2 



i=l 

2 1 



i " 9 i ,l 

JYT Z ^ ' /T Z TJ -^ ' 



= log 



9 ' 9 



i=l 

2 2 / i n 

■ Z^ & _ G 



<7 2 <7q 



i=\ 



1 " 2 1™ 

TIG /T ri r * 



i=l 



2 2/i n \ 9 1 n 



'0 



1=1 



1=1 



It is now easy to see that 



2 2 

inf — Ul:n(-Xl:n 0) — £l:n(-X"l:n|#o) = — Ul:n(^l:raFn) _ ^l:n(-X"l:n|^o)] 

8eB(8 ,8) n n 



= -bn 



2 2 / i n \ 91™ 



2^2 



c„cr 



n u a 



II 



i=l 



at n 



-b n (6n) + -» 



1 



2 2 / i ™ \ 9 

ct2 u^ ) n U 



'0 



H — o^ 






(j^ n 



(6.3) 
(6.4) 



where R„ " '> because of the Law of Large Numbers and Lemma 7.6. Thanks to Lemma 7.3 
we thus find that there exists C € R+ such that 



,J& A l^ x -^ ~ iUx,M] > (log | + 1 - | 



— 2 (C||0 n - Soil - ^n 
u n 



We now choose k > and <5 > small enough so that 



o* log^ + 1-^ >-« 



3(k + C£ 2 ) _1 
2(^-5) ^"2 £ - 



(6.5) 
(6.6) 
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Thanks to (6.5) and the definition of 9 n , we can now write that 



inf -[luniXunld) - h:n(X U n\0o)} > J (« + C\\9 n - 9 Q \\ 2 - R n ) 

0gb(0q,5) n (J n 

>-\{K + C8 2 -R n ). 



Since for any n large enough 



\R n \^l^ + C6 2 ), 



we find via (6.6) that for any n large enough 

inf _ -[l 1:n (X 1:n \e) - h: n (X 1:n \e )] > ~^J (« + C5 2 ) 

3(k + C8 2 ) 1 

" 2(a 2 -5) " 2- 

We just proved that for any e > 0, we have a.s. for any n large enough 

^ inf -\h:n{Xi.. n \e) - li; n (X 1:n \e )} > -\e, 

8eB(8 a ,6) n 2 



which immediately implies 



tt(0) exp[J 1: „(X 1:n |0) - Zi: n (X 1: „|0o)] d0 > e~ ™ e \ ir(9) M. (6.7) 

B(6 ,S) Jb(6 ,S) 

Conclusion. Let now e > and 5 > small enough so that a.s. for any n large enough (6.1) 
and (6.7) both hold. We have a.s. for any n large enough 

Sbh8q,S) jgj cMh:n(X 1:n \9) - h :n (X 1:n \0 o )] d6 f BHeOtS) n(O)d0 _^ 
I B{ e , S ) < e ) eMh:n(X 1:n \9) - lun(X 1:n \6 )} d9 ^ j B{ ^ 5) n(6) dtf ' ~* ' 

which ends the proof. □ 

Proof of Theorem 2.3. Because the posterior distribution of 9 in the pseudo-problem, 7r*(-|Xi : „), 
can be written as 

<(0|X 1:n )cx7r(0)expK :n (X 1:n |0)], 
the posterior density of t* = n^(9 — 0*) £ R 3 can be written as 

K(t\Xi-.n) = c-^ei + n-h)eMil,n{Xi-. n \e* n + n-h) - it. n (x 1:n \d* n )} 

where 

C n = f 7:(6* n + n-h)eiq ? [ll. n (X 1 .. n \e* n + n-h)-lt n (X 1 : n \d* n )}dt (6.8) 

Jr 3 

Denoting 

9n (t) = w(e* n + n-h)e XP [l*l:n(Xl:nK+n*-h) - l* 1:n (Xl:n\0t)] 

-Tr{9 )e-i t ' I( - e ^ t 1 (6.9) 
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to prove (2.9) it suffices to show that for any ^J k ^ ko, 

\\t\\ k \g n (t)\dt^0. (6.10) 



Indeed, if (6.10) holds, C n -^ 7r(6 , )(27r)i |/(6» )|~ = (k = 0) and therefore, the integral in (2.9) 
which is dominated by 

C" 1 / \\t\\ k \g n (t)\dt 

JR 3 

+ f \\t\\ k C- 1 7r(0o)e-' t ' /(eo) *-(27r)-'|7(0o)|'e-' t ' /(8o) * dt 

Jr 3 

also goes to zero a.s. 

Let < 5 to be chosen later, and let ^ k ^ ko. To show (6.10), we break R 3 into two regions 

Ti(S) = B c (0,5n?d n ) = {t : \\t\\ > 5n?d n ] 

T 2 {8) = B{0,SnU n ) = {t : \\t\\ < SnU n } 
and show that for i = 1, 2 

/ \\ t \\ k \g n (t)\dt^0. (6.11) 

Proof for i = I. Note that J T , S) \\t\\ k \g n (t)\ is dominated by 

||i|| fe 7r(^ + nh) exp[l* Un (X 1:n \6* n + n~U) - l* 1:n (X 1:n \0* n )] dt 

Ti(S) 

+ f ||i||^(0 o )e-^' 7(eo) *di. 

The second integral trivially goes to zero. For the first integral, we observe that it can be rewritten 
as 



n ! / n7\\e-6U*n(0)e X p[n :n (X 1:n \0)-lt n (X 1:n \e*)]de. 

The strong consistency of #* (see Theorem 3.3) implies that a.s., for any n large enough 

||^-0 o ||<i<ta„. 

From this, we deduce that a.s., for any n large enough, B c (8^,Sd n ) C B c (8o, \8d n ) and thus that 
the first integral is dominated by 

n^ f p-PjMO)w[n m (Xi™\0)-UXi:n\fr n )]M. 

JB"(e ,^5d n ) 

Recalling that n* ~ n, Proposition 7.11 with p n = d n implies that there a.s. exists e > such 
that for any n large enough and any 9 £ B c (6q, ^dn) we have 

lt. n (X 1:n \6)-l* 1:n (X 1:n \e* n )^-end 2 n . 
It follows, using (2.6) that, a.s. for any n large enough the first integral is dominated by 

*P exp(-end 2 n ) [ \\0 - 9* n \\ k Ti(e) dt = n^ exp(-end 2 n ) • 0(1) 
Je 

sCn^n- dlog ™- 0(1) ->0, 
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fc+1 

n 



since by (2.6) we find that nd^ ^ (logn) 2 for any n large enough. Hence (6.11) holds for i = 1. 

Proof FOR i = 2. We first recall the multivariate Taylor expansion for a function g (k+l)-times 
continuously differentiablc within a neighbourhood of y £ R™. With the usual differential calculus 
notations 



we have 



D a g(y) ■ h™ = J2 



d a g 



l^ii )...,i ^n 



9n • • • dj, 



■(y) • h h ---hi 



k 

?(*) = E ~i Da 9(y) ■ (x - y) {a) + Rk+i{x) 

*— ' IT' 



(6.12) 



where 



^fc+i(aj) 



(k + l) lJ0 



(1 - s) k B k+1 g(y + s(x - y)) ■ (x - y)( fe+1 ) ds. 



(6.13) 



Before expanding the log-likelihood over T2(S) in a such a way, we first have to make sure it 
is diffcrcntiable over the correct domain. Indeed, the strong consistency of 0* (sec Theorem 3.3) 
implies that a.s., whatever So > 0, for n large enough, 

\\9* n -6o\\ <S d n . 

For d chosen small enough, since t £ T2(5) implies 

\\e-e*j<6d n 

it follows from the triangle inequality that a.s. for n large enough, 

||0-0o|| < {S + S)d n <d n . 

A.s. for any n large enough, t € T 2 (S) hence implies 9 G B(9o, (S + 5o)d n ). We choose So and 
S small enough so that S + Sq < 1. This way, 9 H> l*. n (Xi :n \9) is guaranteed to be infinitely 
continuously diffcrcntiable over B(9q, (S + 5o)d n ) C B(9o, d n ). 

Now expanding the log-likelihood in a Taylor series for any n large enough, and taking advan- 
tage of the fact that l*- n (Xi m \(%,) = 0, we define B*. n (-) the symmetric matrix defined for u £ D n 
by 



r «2;* 



B* 1:n (9) 



d 2 l* 1:n (X 1:n \9) d 2 lt.n(Xl:n\9) d%* : „ (X 1:n |0) 



djdj 



djdu d^/da 2 

d% n (X 1:n \9) d 2 l* 1:n (X 1:n \9) 



dudi 



and write that 



l l:n( X l-n\9) ~ V[, n (X 1:n \9* n ) 



){e-9*J 

hi?3,n(«) 



„ duda 2 

d 2 i! n (x 1:n \o) 

da 2 da 2 



b*iM))V-K) 



(6.14) 



(6.15) 



where 



Rs,n(0) = 3, J (1 - s) 2 n% n (X 1:n \9* n + s(9 - O • (9 - P n )W ds. 



(6.16) 
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Lemma 7.12 allows us to write that a.s. there exists a constant C £ R*, such that for any n large 
enough, for any t £ T 2 (8) 

l* 1:n (Xl:nK + U-h) - l* 1:n (X 1:n \0* n ) = ~\t' (jl' 1 B*._ „(£)) t + S n (i) (6.17) 



where 



|5„(t)| < Cn"' ■ ||*ll 



(6.18) 



From (6.18), we obtain that for any t £ T2(S), S n (t) —4 0. Because of Lemma 7.10, we have 
n~ 1 Bl :n (6* l ) -^4 I(0q), and it follows immediately that for any t £ T 2 (8), 



and thus that 



From (6.18) we also obtain 



9n(t) ^ 0, 

\S n (t)\ s: C5d n \\t\\ : 



Lemma 7.10, combined with (2.6), (6.17) and the positivity of I(0q), ensures that a.s. for any n 
large enough 

iff —Id* /£* 



|S n (i)K / \n- l Bl, n {6l)) t, 

so that from (6.17), a.s. for any n large enough 

exp[ll :n (X l:n \e* n + n~U) - l* 1:n (X 1:n \e* n )} ^ e -^( n " lfl >*-^»' < e"* 4 ' W. (6.19) 

Therefore, for n large enough, ||i|| fe |g„(t)| is dominated by an integrable function on the set T 2 (8) 
and (6.11) holds for i = 2 which completes the proof. □ 

6.2 Proofs of Section 3 

Proof of Theorem 3.1. From (6.4), it is easy to see that 

-[h:n(X Un \e) - l 1:n (X 1:n \K)] < -[h :n (X Un \e) - h; n {X 1:n \0 O )} 

n n 



< -b n (0) + 



2 1 



o-i-o- 2 l 



° 2 °l 









a n 



For any 9' £ 6 and r > 0, let B(0', r) = {0, ; \\0' — 0\\i < r}. It is now obvious that 



-[h:n(X 1:n \B(0 , ,r))~l 1 .. n {X 1:n \K)} 



^ sup {— b n (0)} + sup 

0£B(8',r) 6eB(8',r) 



T 2^2 



1 " 



+ Sup < — 2 > • SUp 

8£B(0',r) l°~ J 8£B(6',r) 



1 ™ 



(6.20) 
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Lemma 7.6 now ensures that 



sup 

0&B(0',r) 



1 - 

Tt ' * 



1=1 



and a being bounded away from ensures the boundedness of sup ^ — (■ which implies 

8eB(8',r) V a 



sup 

0£B(0',r) 



sup 

0£B(0',r) 



1 ™ 
n *■ — ' 



i=l 



0. 



Since a 2 is bounded away from 0, taking advantage of the Strong Law of Large Numbers, we also 
obtain 



sup 



2^2 



a- a. 



Tt * ' 



We may thus rewrite (6.20) as 



-[h:n(X 1:n \B(6',r))-l 1:n {X 1:n \K)}^ sup {-b n (6)} + i?„, 

n 0£B(0'.r) 



(6.21) 



where R n —^ 0. 

Assume now that 9' ^ 9q, then we have 

sup \b n (0) - b{0')\ ^ sup \b n (9)-b n (9')\ + \b n (6')-b(9')\. (6.22) 

0GB(0',r) 0£B(0',r) 

Lemma 7.5 (sec (7.9)) ensures the existence of a r small enough, say r = r(6'), such that 

sup \b n {9)-b n {9')\^-M9'), (6.23) 

0£B(0',r(0')) 4 

uniformly in n. For n large enough, that same Lemma 7.5 (see (7.10)) also guarantees that 

\b n {0')-b{0')\^\b{0'). (6.24) 

Adding inequalities (6.23) and (6.24) together and combining the result with (6.22), we deduce 
that for any n large enough 

sup \ bn (9)-b(9')\^h(6'), 

0eB(0',r(0')) & 

i.e. 



sup {-b n {9)} ^ --b(9'), 

0<EB(0',r(0')) Z 

which finally gives together with (6.21) 

W?0 O , F (]imsup -[h: n (X Un \B(9\r(9'))) - l 1:n (X Un \K)} ^ -h(9')) =1. (6.25) 

Since Lemma 7.5 ensures that b{9') > for any 9' ^ 0o, the previous statement implies 

W ^ O , V(3n(0') e N, Vn > n(9'), h.. n {X 1 .. n \B(0',r(0'))) - l 1:n (X 1:n \K) < -1) = 1. (6.26) 
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For a given S > 0, let us now define K(S) = K \ B(9o, S). K{8) is obviously a compact set since 
K itself is a compact set. By compacity, from the covering 

|J B(9',r(6'))DK(6), 

8'£K(6) 

there exists a finite subcovering, i.e. 

m{8) 

3m{8) G N, (J B(0j.,r(0$)) D tf(tf). 

In particular, (6.26) holds for 9' = 6j,j = 1, . . . ,m{5). Let us define 

n o(<$) — max n(6A. 

j=l,...,m(5) J 

We may now write 

V<5 > 0, 3n (5) G N, 3m(6) G N, Vj = 1,. . .,m(5), 

P(Vn>n (<5), «i : „(X 1:n |B(^,r(^)))-/i :n (X 1:n |is:) < -l) = 1, 

which we turn into 

V<5 > 0, 3n Q (S) G N, 3m(<5) G N, 

P (Vn > n (<5), Vj = 1, . . . ,m(5), h sn (X 1:n \B(e' j ,r{e' j ))) - l 1:n (X 1:n \K) < -l) = 1, 

thanks to the finiteness of m(5), and finally into 

V5 > 0, 3n Q (S) 6N,P (Vn > n (S), l 1:n (X 1:n \K(S)) - l 1:n (X 1:n \K) < -I) = I, 

because of the covering 

m(6) 

{jB(9> r r(0>))DK(5). 

3=1 

Let us now sum up what we have obtained so far. We proved that 

VS > 0, 3n (S) G N, P (if Vn > n (<T), Zi : „(X 1:n |0) - l 1:n (X 1:n \K) > loge" 1 , then g if (5)) = 1, 



3a = e- 1 G]0, 1[, V5 > 0, 3n (<5) G N, P (if Vn > n (S), 6e K n {a), then ||0-0 o ||i < $) = 1, 
that is to say 

3oG]0, If, P( lim sup 110 - 6> ||i = | = 1. 

D 

Proof of Proposition 3.2. In this proof || • || will refer to the usual Euclidean norm. Reindexing 
whenever necessary, we also assume that the observations ti are ordered, and we denote 

t = (t 1 ,...,t n ), X = (X 1 ,...,X n ), [to = (Kvo,ti),-.-,n{r}o,tn)), 
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1 n 1 n 

7V ,„ = sup{i, U < M } = -};l\ tii+00 \(uo), N n = sup{z, U < u n } = - ^ t [ti:+oo[ (u n ), 

f (0,...,0, / 5 +7o*w„+i,---,/3o+7oiiVo„,0,...,0), if JV„ < JV .„ 
C= < (0,...,0), ifiV n = 7V , n , 

[ (O,...,O,j0o+7o*JV o ,»+i> •••>#>+ 7o*iv n ,0,...,0), ifiV„>iVo,„ 

Let Q be the linear space spanned by the 2 linearly independent n-vectors 

vi = (1,...,1,0,...,0) v 2 = (t 1 ,...,tjv n ,0,...,0) 

(both of which have their last n — JV n coordinates valued to zero), and denote Q the orthogonal 
projection onto Q. 

Let Q + denote the linear space spanned by v±, v% and Mo and denote Q + the orthogonal 
projection onto Q + . Observe that Q + is also spanned by vi, V2 and £• 

Finally, denote fi* the orthogonal projection of X onto Q + and m the closest point to X in C/ + 
satisfying the continuity assumption of the model, i.e. 



We have 



Thus 



which leads to 



/1* = Q + X, £= (nirjnjh), . . . , n(rj n ,t n )). 

\\X M *|| 2 + || M * - m|| 2 = ||X - MJ| 2 sc ||X - M o|| 2 , 

\\x - Moll 2 - IIm* - Moll 2 + Ma** -tA\ 2 ^\\x- Moll 2 , 

||m* -Moll 2 -2(m* -mo,M~mo) + ll/"-^o|| 2 sC ||m* -Mo|| 2 - 
||m-mo|| 2 < 2(m* -mo,M-Mo) < 2||m* — A*o|| ■ Ma* — A*o ||, 



llM-Moll ^2||m*-Mo|| ^2||Q + e||. 
Our aim is to show that a.s. 

||Q+e|| = 0(logn). (6.27) 

If (6.27) held, then we would have a.s. ||m — Moll =0 (logn) i.e. a.s. 



^2(KVn,U ~ KilOiU)) 2 = 0(log 2 n) 

i=l 

Hence, a.s. for any open interval / C [u, u] we would have 

n 

^T(KVn,ti - n{r]0,U)f l/(*i) = o(iog 2 
»=1 

This would immediately imply the desired result, i.e. that a.s. 

min \n(f) n ,ti) - (Jt(r]o,ti)\ = O n^'logn 



2 \ 

n) 



since a.s. 



n -. n 

O (log 2 n) = y^(v{rf n ,ti - fi(vo,ti)) ti(t t )^n- min \fJ,(ff n ,U) - ft(vo,ti)\ •-V'lj(ti), 

r — *iG^, «^n n — ' 
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where (see Assumption (Al)) 

i j2 ii(ti) = y dF„(i) -> y dF(i) = y /$ & > o. 

Let us now prove that (6.27) indeed holds. We consider the two following mutually exclusive 
situations. 
Situation A: £ = (0, . . . , 0). In this situation 

IIQ + 4ll = IIQfll, (6.28) 

and Cochran's theorem guarantees that ||Q£I| 2 ~ X 2 (2) for n Js 2. Hence, via Corollary 7.7, a.s. 

||Q£||=0(logn), (6.29) 

and (6.27) follows from (6.28) and (6.29). 
Situation B: ( ^= (0, . . . , 0). Since 

KC;£)I .r/ n 2\ 

-TTjj A(0,cr o ), 

we also have, via Lemma 7.7, a.s. 

I (CO I 

IICII 

Notice that (6.27) follows from (6.29) and (6.30) if we manage to show that a.s. 



= 0(logn). (6.30) 



iiQ + eKo(i).^iigeii + ^^). (6.3i) 

It thus now suffices to prove that a.s., for any g G Q 

|(C,ff)| = IKII 11.911 -o(l), (6.32) 

where the o(l) mentioned in (6.32) is uniform in g over Q (i.e. a.s. £ is asymptotically uniformly 
orthogonal to Q), for (6.31) is a direct consequence of (6.32) and Lemma 6.1 whose proof is found 
in Feder (1975). 

Lemma 6.1. Let X and y be two linear subspaces of an inner product space £ . If there exists 
a < 1 such that 

V{x 7 y)eXxy 7 \(x,y)\^a\\x\\ \\y\\, 

then 

II* + y|| ^ (1 - o)- 1 (||a:*|| + ||y*||), 

where x* (resp. y* ) is the orthogonal projection of x + y onto X (resp. y). 

Observe that, as a consequence of Assumption (Al) and Theorem 7.1, the three following 
convergences are uniform in u over [u, u] for k = 0, 1, 2, 

1 n r u r u t u 

-X>?l[ti, +<*,[(")= / t k dF n (t)^ t k dF(t)= t k f(t)dt. (6.33) 
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We have a.s., for any g(4>) = (coscj))vi + (sin</))t>2 € Q. with (f> € [0, 2ir 

l(C,5(0))l = 



^ (A) + loti) (cos (j> + tj sin ift) - y^(/3 + 7nii)(cos + i 2 ; sin < 
»=i »=i 



< (max(|u|,|u|) + l) 



N„ N ,n 

£|A)+7o*i|- 51 IA}+7(A 



<(max(|w|,N) + l)-||C||i 

< (max(|«|, |u|) + 1) ■ HCll • n^\N n - N , n \i 



<(max(N,|u|) + l).||C||-n= 
i.e. we have a.s. for any <j> £ [0, 2-k] 



1 n i n 



KC,5W))| = n»||C||-o(l), 



(6.34) 



thanks to the strong consistency u n — — » uq (see Theorem 3.1) and the uniform convergence 
mentioned in (6.33) with (k = 0). Observe that the o(l) mentioned in (6.34) is uniform in <fi over 
[0, 2ir]. We also have a.s. for any <f> £ [0, 2ir] 

1 1 ™ 

-||.9(0)H 2 = -*S2(cos(/) + Usin4>) 2 l [t +oo[ (u„) 

-. n 1 n 

= - X! •"•[**. +oo[(«n) cos 2 ^ + 2 - X! **l[tt, +oo[(u«) cos (/> sin 
=1 

1 " 
-^*il[ti,+oo[(wn)sin 2 



; = 1 



; = 1 



■^> cos 2 / /(t)dt + cos0sin0 / 2tf(t)dt + sm z (j) ff(t)dt, 

J u J u J u_ 

once again making use of the strong consistency u„ '> uo (see Theorem 3.1) and taking advantage 
of all three uniform convergences mentioned in (6.33). We thus obviously have a.s., uniformly in 
4> over [0, 2n] 



1 f u ° 

-HsWOII 2 ^ / (cos0 + <sin0) 2 /(i)dt. 



(6.35) 



The limit in (6.35) is a positive and continuous function of </>, and is hence bounded, i.e. there 
exists m > such that we have a.s. 



1 



-|| ff (</>)|| 2 ^m + o(l), 



(6.36) 



15(0)11 



0(n->), 



(6.37) 



where the o(l) mentioned in (6.36) and the 0(n 2) mentioned in (6.37) are uniform in <fi over 
[0, 2-k}. 

Combining (6.34) and (6.37) together, we have a.s. for any <j> £ [0, 2-k] 



l(C,<?W>))l = IICII Il3(0)ll-o(i), 



(6.38) 
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where the o(l) mentioned in (6.38) is uniform in <fi over [0, 2ir]. 

Hence, we have a.s, for any r € R+, and any <f> € [0, 2tt], now denoting g(4>) = (rcos<f>)vi + 
(rsin</>)f2 and applying (6.38) to r~ 1 g(4>) 

\{C,9(<f>))\ =r\(C,r- 1 g(<f>))\ =r-\\(\\ Wr' 1 g(<f>)\\ • o(l) = ||CII llffWH-o(l), 

where the o(l) mentioned is uniform in (f> over [0, 2-7r] and does not depend on r. 

We immediately deduce that a.s. (6.32) holds i.e. a.s. ( is asymptotically uniformly orthogonal 
to Q, which completes the proof. □ 

6.3 Proofs of Section 4 

Proof of Proposition 4-1- We proceed as announced. 
Step 1. We first prove that a.s. 

3N e N, Vn > N, u* n g D n . 

Let us notice that anything proven for the problem remains valid for the pseudo-problem. Because 
n* ~ n, we have a.s., thanks to Theorem 3.3 and conditions (2.6), asn-> +oo 

ni (log -1 n)-(u* n -u ) = 0(1), 
n? (log - n) • d„ — > +oo, 

and thus deduce from the ratio of these two quantities that 

w£ - "0 a.s. 



d„ 



0. 



and this directly implies the desired result. 

Step 2. Let -A*. n (-) be the column vector defined for u G Z? n by 



At. n (0) = 



dl* 1:n (X 1:n \9) 



o~ 



di* 1:n (x 1:n \e) 



On 



dit. n (x 1:n \e) 



da 2 



(6.39) 



Step 1 allows us to expand a.s. A*. n (0*) around 9q using a Taylor-Lagrange approximation 

= ^ : „(^) = AJ :n (e )--B 1 *:„(fln) 



where 9 n is a point between #* and 6*o (see (6.14) for the definitions of -B*.„), and rewrite it as a.s. 

1 



: Bt n (6 n )-n*> 



o)=n*-iA* 1:n (e ). 



Since 0* — > 6$, we also have 9 n — > 9q and using both Lemmas 7.9 and 7.10 we immediately find 
that asn-> +oo 



I(0o) ■ n*- 



o ^M(0,I(9 )) 



which means, remembering both that n* ~ n and that /(#o) is positive definite and thus invertible 
that as n — > +oo 



^(^-^^^(o,!^)- 1 ) 



n 



22 



Proof of Theorem 3. 3. We now prove that 

l|5£-°oll =0(n-*logn 



Variance of noise a 2 . Observe that 
1 ™ 

&n = ~ E i Xi ~ ^" ( fi ~~ W„)l[t i , +00 [(u n )] 
i=l 

1 ™ 

= ~E b° ' (*' _U o)l[t i ,+oo[("o)-7« ■ (<i -Un)t[ ti , +00 [(Un) + &] 

= - E *?(*o + - E *)& + - E & ( 6 - 4 °) 

i— 1 i— 1 i—1 

where wc denote for i = 1, . . . , n, 

Vi{rj) = 7o ■ (*» - wo)l[i i ,+oo[(wo) - 7 ■ (*i - w ) ]1 [t,, +<»[(«)• (6-41) 

We have 

SUp|^i(?7„)| = SUpko • (^-Uo)l[t,,+oo[(^o) -7n ' (*» -W„)l[t il+ oo[(Wn)| 

iGN ieN 

< 1 7o -%| • sup|(£j - uo)l[ ti , +0 o[(uo)| 

i£N 

+ \%\ -SUp|(ti -« )l[t 4 ,+oo[(«o) - (*» -«7i)l[ti,+oo[(«n)| 

iGN 

= 0(7o-7n) + l7n|0(u -u„), (6.42) 

using straightforward dominations and Lemma 7.2, so that in the end, thanks to the previous 
results we have a.s. 



It is thus easy to see that a.s. 



sup \vi(rf n )\ = O (n 2 l ogn ] . (6.43) 



1 n 

- E "iiVn) = O (n- 1 log 2 n) = O (n - ' logri) , (6.44) 



n . 

2—1 



and also that, via Corollary 7.7, a.s. 

- E V *®»K* = - E ^«) • 0( lo S«) = O (n-i logn) . (6.45) 

i=l \i=l / 

From the Law of the Iterated Logarithm (see Breiman, 1992, Chapter 13, page 291) we have a.s. 
-E fe 2 - CT o) = O (n-*(loglogn)4) = O (n"* logn) (6.46) 

8=1 

and the desired result follows from (6.44), (6.45) and (6.46) put together into (6.40). □ 

Proof of Theorem 4-2. To finish the proof, we need to show (4.3) i.e. that 

°l - &l* = °p (n~ 
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We use the decomposition (6.40) 

1 n (-. n -. n 

d n = - E ^(^) + - E ^^ + - E ^> 

i— 1 z— 1 z— 1 

where ^(ifo) = 70 • (** - Uo)l[t 4i+00 [(uo) - % ■ (U - u n )t[ ti:+00 [(u n ). 
Having proved in Proposition 4.1 that 

7* - 70 = Op (n~^J , u* n - u = P (n~^) 

we add these relationships to those from (4.2) and find that 

% - 7o = Op (" _T J , u n - u = Op (n 

We now use (6.47) together with (6.42), we are able to write 

sup \vi(rf n )\ = Op (n~* ) . 

It is hence easy to see that 

1 " 

-E^(^) = P (n- 1 ) = 

i—l 

and also that 



op I n 2 



(6.47) 



(6.48) 



2 2 / \ / 

- E v i(Jh)& = - [ E ^«) ' 0p ( 1 ) = ° p ( n ^ 

n i=l n \i=l / 

which once both substituted into (6.40) yield 



1 " 

5* = -E£ 

n * — » 



+ Op In 



What was done above with the problem and ct„ can be done with the pseudo-problem and a^* 
without any kind of modification so that 



;2* 



1 ™ 



+ Op 71 



!=1 



We observe that 



_. n 1 n* 

71 ■'--' n* *-** V / 

i=l 

n* _. n 

£e? + ~ E £+<*(»-*) 



z- 


=1 




"1 




1 " 


n 




71* 



71* — n / 1 



71 \ 71' 



E^ 2 



2— n*+l 

n — n* 



n \ 7i — 7i 



i n \ 

— E £+<*(""*) 



i=n*-\-l 



(TS + Ogln* 5 + 



1 \ \ n — n 



Oq + Or ( (« - n*) 2 )) +opU 
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□ 



using the Central Limit Theorem, and in the end we get 

°l - °l* = U ■ Op ( n *~^) + ^—^- ■ P ((?i - n*y?) + o P (V 

= o(l) • Op [n~^J +n~5 • Op ( J +o P (n~^J 

— Op (n~^J + n ~3 • Op (o(l)) + Or (n~?) = 0p fn - ') • 

7 Technical results 

Theorem 7.1 (Polya's Theorem). Let (g n )n£N be a sequence of non decreasing (or non increasing) 
functions defined over I = [a, b] C K. If g n converges pointwise to g (i.e. g n (x) —> g(x) as 
n — > +00, for any x £ I) and g is continuous then 

sup \g„{x) - g(x)\ —— — > 0. 
xei n^+00 

Proof of Lemma 7.1. Assume the functions g n are non decreasing over / (if not, consider their 
opposites ~g n )- g is continuous over I and thus bounded since / is compact, g is also non 
decreasing over I as the limit of a sequence of non decreasing functions. Let e > and k > 
such that 

3a = a < . . . < a k = b G I k+1 , Vz = 0, . . . , k - 1, g(a i+ i) - g{ai) < e. 

Now let x G I and let i G N such that a, ^ x ^ a»+i. Since g n and g are non decreasing, we find 
that 

g n (x) - g{x) < g n {a i+ i) - g(a t ) ^ g n (<k+i) - fffai+i) + e, 
ff n (x) - g(x) ^ ff n (oi) - sK+i) > 9n{ai) - g{ai) - e. 

The pointwise convergence of g n to g and the finiteness of k together ensure that 

3N Q G N, Vn > JV , Vi = 0, . . . , fc, |flf„(oi) - ff(oi)| < e, 

which implies with both of the inequations mentioned above that 

3N G N, Vn ^ JV , Vx G I, |<7„(z) - 3 (x)| < e. 



□ 



Lemma 7.2. Lei k G N*, i/iere exists a constant C G R+ suc/i i/iat /or anj/ {u,u') G [u, u^ 



sup_ |(* - U ') fc l[t,+oo[("') - (* - «) fc l[ t ,+oo[HI = C|« - «'|. (7.1) 

Proof of Lemma 7.2. For any (u,u') £ [w, w] 2 we have 

SU P _ \(t - u')*l[t, +«,[(«') " (« " «) fc l [t , +oo[ H| < SU P _{|(t - U'f - (t- U) k \t [t:+00[ (U')} 
££ [it, u] t£ [u, u] 

+ sup_{|<-u| fc |l [ti+oo[ (ii')-l[t,+oo[(w)|}. 

£6 \u, u] 

(7.2) 
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The mean value theorem guarantees that there exists v between u and u' such that 

(t - u') k -(t- u) k = -k(t - vf-^u' - u). 
We thus have 

sup_{|(i - u') k -(t- u) k \t [t . +oo[ (u')} < sup_ \(t - u') k - (t- u) k \ 

^k\u-u\ k ~ 1 \u-u'\. (7.3) 

Because \t — u\ ^ \u' — u\ whenever |l[ t-+00 [(u') — l[t,+oo[( u )l ¥" 0, we also find that 

sup_{|i - u\ k \t %+oc[ (u') - l [t , + oo[(«)|} < \u-u'\ k < lu-u'Hs-al*- 1 . (7.4) 

t£i\u, u] 

And now (7.1) is a simple consequence of (7.2), (7.3) and (7.4). □ 

Lemma 7.3. For any rf £ R x [u, u], there exists C £ R+ such that for any r\ £ R x [u, u] 

sup \n(ji,t)-M,t)\<C\\ri-rfl (7.5) 

tG \u, u] 

Proof of Lemma 7. 3. We have indeed 
sup_ \fi(ri,t) - fj,(r)',t)\ = sup_ I7 • (t-u)l[ tl+00 [(u) - -y'(t - u')l[ ti +00 [(u')\ 

< sup_ |[7-7'](t-«)l[ t ,+oo[(«)| 

t€i\u, u] 

+ sup \-y'[(t-u)l[ t , +00 [(u)-{t-u')lit,+oo[{u')]\ 

te[u,u] 

^ |«y _ -y J . Su p |/j — y| 

+ It'I • sup_ |(*-ii)l [t)+oo[ (u) - (i-ii')l[t, +0 o[(OI 

< l7-7'|-|w-u| + | 7 '|- sup_ |(t-u)l[t, +oo[ (u)-(*-tt , ) 1 [*,+oo[( u OI- 

And now (7.5) is a simple consequence of Lemma 7.2. □ 

Lemma 7.4. Let A C R X [w, «] be a bounded set. Then, 

Ve > 0, 3m(e) £ N, 3%, . . . , r? ro(e) G A, 

Vr7,r/GA 3j,j' G {1,..., m(e)}, sup |[/x(r7,t)-/x(rj',t)] - [/*(%,<) ~ Mfai'j*)] | < e, 

Proof of Lemma 7.4- It suffices to prove the following claim 

Ve > 0, 3m(e) G N, 3 m , . . . , Vm{t) £ A, 

V77 G A, 3j G {1, . . .,m(e)}, sup |/i(r/,i) - n(r)j,t)\ < e. 

te[u, u] 

and then use the triangle inequality. To see that the claim holds, it suffices, thanks to Lemma 7.3, 
to exhibit a finite and tight enough grid of A such that any point in A lies close enough to a point 
of the grid. The existence of such a grid is obviously guaranteed since A C R 2 is bounded. □ 

Lemma 7.5. Recall the definition of b n given in (6.2). Let 

W) ={$- 1 - lo ^) + ^£ Mvo,t) fa, t)f f(t) at. (7.6) 
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Then, under Assumptions (Al)-(A4), 



b n {9) > 0. 

b{9) ^ 0, with equality if and only if 9 = 9q. 
b n (9') — > b n (6), uniformly in n, as 9' — > 9. 
b n (6) -> b(9), as n — )• +oo. 



(7.7) 

(7.8) 

(7.9) 

(7.10) 



Proof of Lemma 7. 5. We will prove each claim separately. 

Proof of (7.7). That b n (9) ^ is trivial since the first term in (6.2) is non negative (having 
x — 1 — logrr Js with equality only if a; = 1). and the second term in (6.2) is obviously non 
negative too. 

Proof of (7.8). That b(9) ^ is again easy enough to prove, both terms in (7.6) being 
trivially non negative. If 9 ^ 9q then either a 2 ^ a 2 , which implies the first term is positive, or 
A^o, 7^ f-iVi ') which implies the second term is positive (since / is assumed positive on \u, u]). 
Hence if 9 ^ 9 then b(9) > 0. That 9 = 9 implies b{9) = is of course straightforward. 

PROOF of (7.9). We first observe that 



1 n i n 

- y]0(?7o, U) - fi(v', U)f y2(fj,(r] , U) - n(v, U)) 

i=\ i=\ 

1 - 

- X] [ 2 ^(^ ' **) ~ M 77 '' **) ~ ^C 7 ?' **)] ' t^' 7 ' *») ~ ^ i ( 77 '' **)] 

1 - 

^ - IT. |2/i(?7o, U) - (J,(r)', U) - n(r), U)\ ■ \n(r), U) - (j,(r)', U)\ 



^ sup_ \n(r]o,t) - fj,(rj,t)\ + sup_ \fi(rj',t) - /x(r/,t)| • sup \n(r)',t) - fi{i] t t)\ . (7.11) 

WEfu, u] t£\u, u] J t(E[u, u] 

As 9' — > 9, the convergence of the first term of b n to the first term of b is obviously uniform in n 
since this part of b n does not involve n at all. As 9' — > 9. via Lemma 7.3, we also obtain 

sup_ \fj,(v',t)- h(t}, t)\ ->0, 

which ensures that the second part of (6.2) converges uniformly in n thanks to (7.11). 
Proof of (7.10). Thanks to Assumption (Al), it is easy to see that 



-^2[v(r)o,U) - v(v,ti)} = / \it(r)o,t) - n(r),t)] dF n (t) 



\ft{vo,t) - M(r/, t)] 2 dF(t) = / [n( Vo , t) - ufa t)} 2 /(*) dt. 



n 

Lemma 7.6. Let A C R x [u, u] be a bounded set, and let j] Q e A, then under Assumptions 
(A1)-(A4), 



sup 



1 - 

-22\M(Vo,ti) - v(ri,tiMi 

n *■ — ' 
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Proof of Lemma 7.6. Let e > 0, n G A, and apply Lemma 7.4 to get the corresponding m(e) G N, 
{771, . . . , rw e )} C A, j, j' G {1, . . . , 711(e)}. We can write with the triangle inequality 

-. n 1 n 

1 
1 - 



n * — ' n 



2—1 



^J sup 

0J')e{l,...,m(e)} 



( 1 ™ 1 1 " 

1 n X}^'' **) ~ til*' ' **)]^ f + e ' « X l& 

I i=l J i=l 



Hence 

sup < - Y)[Ai(7?o,ti) - Kv,ti)]Zi > < sup ^ - y)[/x(77j-,ti) - /»(%':**)]& f 

n 



e ■ 

71 
i=l 

Let us now recall Kolmogorov's criterion, a proof of which is available in Section 17 of Loeve 
(1991) on pages 250-251. This criterion guarantees that for any sequence (li)igfj of independent 
random variables and any numerical sequence (6j)igN such that 

^VarK, 

2 75 < +0°i On -> +°°, 



1—1 



we have 



e^iO^-e^) a^ a 



For each couple {j,j') G {1, . . . , m(e)}, Kolmogorov's criterion ensures that 



1 " 



n 



for the coefficients [fJ-(r]j, U) — /j,(rjji,ti)] are obviously bounded, and it suffices to pick Yi = 
[(i{r)j,ti) — n(r]j' ,U)]£i and bi = i. Having only a finite number of couples (j,j') G {1, . . . , m(e)} 2 
to consider allows us to write 

1 " 

sup -Y]\ii('nj,ti)-K'nj',ti)]€i 3± >0- (7.13) 

(j,i')e{i,->™(e)} n fet 

By (7.13), the first term on the right hand side of (7.12) converges almost surely to zero. The 
Strong Law of Large Numbers ensures that the second term on the right hand side of (7.12) 
converges almost surely to e • (2tt~ 1 o- 2 )^ , and the result follows, since all the work done above for 
(60«eN can bc done again for (— £„) n eN- d 

Lemma 7.7. Let (^i)igN be & sequence of independent identically distributed random variables 
such that for all j £ N, either Zi ~ A/"(0, a 2 ) with a 2 > 0, or Zi ~ X 2 (^) with k > 0. XTien a.s., as 

71 — > +OO 

Z n = 0(logn). 
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Proof of Lemma 7.7. Denote Y n — Z n when the random variables are Gaussian, and Y n = Z n /5 
when the random variables considered are chi-squared (so that Ee 2Yl and Ee _2Yl are both finite). 
We will show that a.s. Y n — O(logn). 

For any e > 0, from Markov's inequality we get: 

P^-V"! > e) =P(n- 2 e 2Y " > e 2 ) < e- 2 n- 2 Ee 2Y \ 
From there it is easy to see that for any e > we have 

+ oo 2 

53 P (n-^e^l > e) = e^—Ee 2 ^ < oo, 

n— 1 

which directly implies via Borel-Cantelli's Lemma (see for example Billingsley, 1995, Section 4, 
page 59) that a.s. 

e Yn = o(n). 

In particular, a.s. for any n large enough, 

Y„ ^ logn. 

What was done with (Y n ) n ^tq can be done again with (— K n )neN so that in the end we have a.s for 
any n large enough, 

— log n ^ Y n ^ log n. 

D 

Lemma 7.8. Under Assumptions (A1)-(A4), for any r/o € K x \u, u], there exists C € P+ such 
that for any n large enough, and for any r\ 



1 ^2[^{m,U)- n{il,U)] 2 > C\\i]-i] \ 



n 



Proof of Lemma 7.8. We have already almost proved this result in (3.3) (see Theorem 3.3). There 
is however a small difficulty since the majoration was obtained for r = (/3, 7) and not r\ = (7, u). 
Let V\ and V% two non empty open intervals of ]u, uq [ such that their closures V\ and V2 are 
do not overlap. We have 



i=l \i=l 

i=l / 

Using the same arguments we used to prove (3.3), we find that there exists C £ M.+ such that 
(remembering the definition of the intercept /3 of the model) 

n / n n \ 

« _1 5Z lKm,U) - Kv,ti)] 2 > min I n~ 1 53 ll Vi(*i),» -1 5Z ll ^(*i) ) -C|7-7o| 2 , 

»=1 V i=l i=l / 



""'XiKlo.ti) -M(l,ti)] 2 ^ min n-^l^^.n-^lyjti) -q/3-/3 | 2 , 

»=1 V i=l i=l / 

and since for j = 1,2 we have 



n 

" -1 51 ^ (**)"► / /(i)dt>0, 
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7o| 2 , 



there exists C € R.+ such that for any n large enough 

n 

t=l 

n 

n- 1 Y, Mm, u) - rtv, u)f > c\p - /3o\ 2 - 

i=l 

Notice now that 

|«-«o| = l7o"Vo-7 _1 )9| 

= l7(7 1 |l/?o-7o7" 1 /3| 

<l7 ( 7 1 |{l/?o-/3| + 1/3 -7o7 _1 /3|} 

<|7 ( 7 1 |{I/?o-/3| + |7- 1 /3||7-7o|} 

<l7 ( 7 1 KI/3o-/3| + l«ll7-7o|) 

< 17^1(1 + max(|u|, |«|)) ■ max(|A, - /3|, | 7 - 7o |). 

From here, since u G [w, u] is bounded, it is straightforward that there exists C €E R?j_ such that 
for any n large enough 



^ Yl t^ 7 ? ' **) ~ ^ tj )] 2 ^ '"i 7 



7o| 2 , 



- 1 J>fa> ) t,-)-/*M0] 2 >ci 



u-uo , 



which ends the proof. 



□ 



Lemma 7.9. Recall the definition of A*. n given in (6.39). Under Assumptions (Al)-(A4) and 
conditions (2.6), as n —> +oo 



n-2A* 1:n (e )AM(0,I{6 )). 



(7.14) 



Proof of Lemma 7.9. We will show that any linear combination of the coordinates of Ai :n (9o) is 
asymptotically normal using Lyapounov's Theorem. Let a € R 3 , ||a|| ^ 0, so that differential 
calculus allows us to write 



(a, A* Vn (e Q )) =qj 



dl*:n(Xl:r 



<:)-, 



a.2 



dit. n (x 1[n \e) 



du 



a3 • 



dll, n {Xl:n\e) 



da 2 



1 n n 

*1 • ~Y [(^ _U o)l[t I ,+oo[(wo) '&] ~ a 2- ~Y [^[ti.+oolCwo) '&] 



1 "' 



— • £ 2 - 1 

_2 Sj x 



8=1 

where we denote, for i = 1, . . . , n 
Zi 



|(t, -«o)l[t(,+oo[(«o) '"1 -7ol[t 4 ,+oo[(«o) -"2J '6+ 2 a 3 ' {cr 2 ■ £? - l} 

(7.1E 



:;n 



Since for i = l,...,n* E[£j] = and E[Cf] = c 2 , we deduce that E [Zi] ~ 0, and hence that 
E[(a,AJ sn (fl )>] = 0. 

Let us now find the expression of Var (a,A^. n (6o)}. Because ^ and £j are independent when 
i =/= j, so are Z, and Zj and we hence write 



Var (a, A* 1:n (0 o )) = o^ >T Var Z, 

4=1 

™* r 

= o-^ 4 ^ j I (ij - u )l[ti, +oo[(wo) ■ "i - 7ol[t(, +oo[(«o) • "2 
1=1 ^ 



Var 6 



-\a 2 ■ Var [a - 2 e 2 - lj 



because Cov [&, {<t 2 £f — l}] =0, and we finally get 

Var(a,vl*.„(0o)) = < 7 (7 4 5I ] |_( il ~ u o)l[t,,+oo[(-«o) • 
»=i *• 



ai — 7ol[ti,+oo[(wo) • "2 



' CT + 4 Q 3 ' 2 



We can hence write 



1 n * 
n*- 1 Var(a,A* 1 . n (9 )) =<7q 2 — ^2 [( tl _ u o)l[t if +oo[(«o) ■ «i - 7ol[t»,+oo[('»o) ■ "2 

i=l 

-2aia 2 •cr ' 2 7o S — 53 ( f * ~ «o)l[t 4 ,+oo[(uo) f 

+ "2 ■o-o" 2 7oj ^T^ 1 !*'^ 00 !^ ) f + a 3' 2 CTo ^ 4 
= (a,/i :n (0 o )a) , 
where we denote 



2 1 



-4 2 
C a 3 



4W 



1 ™ 



[t», +oo[ 



» 



i=l 



(7.16) 



Remark that, by virtue of Assumption (Al), it is easy to check that for any 8 E Q 

r 1:n (9) -> 1 (0), 



(7.17) 



and observe that just like /(0), Ii n {6) is positive definite, since all its principal minor determinants 
are positive. 

Let us now check that the random variables Z% meet Lyapounov's Theorem (see Billingsley, 
1995, page 362) requirements before wrapping up this proof. The random variables Zi are inde- 
pendent and trivially L 2 . We denote V* 2 = X)"=i Var Zi and claim that Lyapounov's condition 
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holds, that is 



36 >0, ^E 



Zi - EZi 



Indeed we have (5 = 1) 



E E 

8=1 



Z; - EZj 



V* 



3 n 



E E 



r* 



z,- 



2+(5 



0(1). 



\'* 



Vax*(a,Al n ($ )) n* ^ 



i - 



n 

•^E e i^i 



The first term of this last product is O (n*~ 3 ) thanks to (7.17), and recalling the definition 
of Zj from (7.15), there is no difficulty in showing that the last term of the product, namely 
-v yy_i IE \Zi\ converges to a finite limit. Indeed we find, using trivial dominations and Assump- 
tion (Al) once again, 



IZI 3 



{{U -%)l[t 4 ,+oo[(Wo) '"I -7ol[f 4 ,+oo[(Wo) '"2} '& + 2 a 3 ' {^O 2 ' & ~ l } 



:«3 



E|Z,| 3 < f I (ti - Uo)l[t il+O o[(wo) • "i -Tol^.+ooi^o) • a 2 | + 

xE(ie,i + k - 2 -ef-i|) 3 
1 - 1 - / 

-^E|Zi| 3 < ~2J ( |(*« -«o)l[t 4 ,+oo[(«o) -ax -7ol[t i ,+ 00 [(wo) ■ a 2 

xE(|^| + |cr - 2 -^-l|) 3 

1 ™ / 

< CK 1 ) ' ~E ( K <1 _u o)l[t I ,+oo[(«o) -"l -7ol[ti,+oo[(wo) -"2 
n 4=1 ^ 

<0(1). 

Lyapounov's Theorem thus applies here and leads to 



2«3 



:«3 



j=i 



V* 



i.e. multiplying numerator and denominator by a we get 



Var'(a,i4J !B (fl )) 



that is 



(a,AJ w (fl„)) _4^ (0>1)> 



n**<a,JT !n (0o)a>* 
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and because of (7.17) we can also write, 



(a '^« (0o)> -^(o,i), 



n*i(a,I(6 )a)i 
which, remembering that a.s. n* ~ n, is equivalent to (7.14). 



D 



Lemma 7.10. Recall the definition of B*. n given in (6.14). Under Assumptions (Al)-(A4) and 
conditions (2.6), as n — > +oo, 



BiJ°o) -^ I{0o), as n -> +oo. 

B*:n(9) —> I(Oo), as -* #o and n -} +oo. 



(7.18) 
(7.19) 



where the asymptotic Fisher Information Matrix /(•) is defined in (2.3). 



Proof of Lemma 7.10. We will prove each claim separately. 

Proof of (7.18). Differential calculus provides the following expressions for the coefficients of 



-B* 



-B* 



7P* 

r-°l:r 



:-Dl:r 



7P* 

:-Dl:r 



:-Dl:r 



11 



13 



22 



2:5 



:::i 



1 " 

4=1 
1 "' 

cr_2 — X! [& + 7 ° ' (*» _ ■"<))%, +<»[(«o) -27- (ti -w)] l[t 4l +«,[(«), 

4=1 
1 "' 

cr_4 — 5Z [& + 7o • (*» -wo)l[t«,+oo[(wo) ~7 • (*t -«)] (*t ~ w)l [tii+00 [(u), 

4=1 
1 "* 
4=1 
1 "' 

-°" _4 7— X [^ +7 ° ' ^ l - u o)l[t s ,+oo[(w ) ~7' (*t -«)] l^.+oolCw): 

4=1 

1 1 "* 

-^ cr_4 + cr_6 — X] fe +7 ° ' (*« -U o)l[t,,+oo[(Uo) -7'(*» - ■")![*., +oo [(«)]' 



The convergence we claim is then a direct consequence of Assumption (Al) and the fact that n* ~ n 
and, depending on the coefficients, either the Strong Law of Large Numbers or Kolmogorov's 
criterion. Notice that 



1 



:Bi :n (0o) - I*:n(flo) 



0. 



where I{. n is defined in (7.16). 

PROOF of (7.19). Wc will show that in fact, as n -> +oo and 9 -> O , 

n* n* 

which will end the proof since n* ~ n. We will consider each coefficient of C*. n (8) in turn, making 
use of Assumption (Al) once again and apply repeatedly the Strong Law of Large Numbers and 
Kolmogorov's criterion as well as Lemma 7.2, whenever needed. 
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ClinWll =(7 (r 2 — X!^ 1 _u o) 2 %i,+oo[(wo) -fT -2 — 53 ( tl - M ) 2l [4i.+oo[(w) 
i=l i=l 

1 "' 
= ((T(7 2 -CT~ 2 ) ' — ^ (*i -Uo) 2 l[ti,+oo[(uo) 

4=1 
/ 1 "* 1 "* \ 

+ CT_2 ' I —^(h -uo) 2 t[ tu+00 i(uo) ~y^fti -u) 2 l[ tl!+00 [(-u) J 

\ i=i »=i / 

= o(l) • 0(1) + 0(1) • (u - wo) -> 0. 
then last equality holding true because of Lemma 7.2. 

Cl*„(^)22 = t7 (T 2 7o^ 51 %,+oolOt)) - °"~V-» 51 1 lU,+oo[(u) 
i=l i=l 

1 ™* 

= (°-(7 2 7o - <r~V) ' — 53 %*> +oo[K) 

i=l 

+ cr _ V ' — $3 %• +~[( u o) - ~z 53 1 [*i,+°°[( t 

j=l i=l 

= o(l) • 0(1) + 0(1) • [{F„. (w ) - F(u )} + {F(u ) - F(u)} + {F(u) - F n , (u)}] 
= o(l) + 0(1) • [o(l) + o(l) + o(l)] -> 0, 

the last equality holding true because of the uniform convergence of i 7 ^* to F over any compact 
subset such as [w, w] (see Assumption (Al), and its Remark 1). 

1 1 "* 

Cr : «(#)33 = 2 CT ~ 4 -0 "~ 6 — 5Z& + 7 ° ' ( <l - u o)l[t I ,+oo[("o) -7' (*t -")![*;, +oo[(^)] 2 

i=i 

-(r.- 4 -^§ £? ) 

i=i 
1 "* 

- cr ~ 6 — 53 [70 ' (^- u o)l[ti,+oo[(wo) -7' (*» - - ")l[t ! ,+oo[(w)] 6 
i=l 

1 ™* 

- cr_6 — 53 [ 7o ' ( <l ~~ "o) 1 !*., +oo[(^o) - 7 ■ (<t - w )l[t s , +<»[(«)] 
?=i 

1 "* 
= o(l) + 0(1) • — 53 £ + o(l) + o(l) ^±> 0, 

4=1 

where the two last o(l) are direct consequences of Lemmas 7.3 and 7.6. 



U 



Those same Lemmas used together with Lemma 7.2, the Strong Law of Large Numbers as well as 
the well-known Cauchy-Schwarz inequality imply that a.s. 

1 "* 

C?:n(0)23 = ^ A -f — ^ [& + 7 ° ' ^ ~ ^Mu. +oo[("o) ~ 7 ' (U ~ u )] 1 [t i .. +oo[(«) 
i=l 

1 "* 

4=1 
1 "* 

-7 X [^""^[ti, +oo[(«) - cT(7 4 7ol[ t! , +00 [(wo)] 6 
=1 

1 "* 

— X! t 70 ' (*» - M o)l[t I ,+oo[("o) -7' (*i _u )] (7_4 7l[t ! ,+oo[('") 



1=1 



71 

1=1 



= o(l) + o(l) -2^0, 

and also that a.s. 



1 r 

Ct. n {0)l3 = <v'— /J [(*i- U 0)l[t il+ oo[(W0)]6 
" f=l 

1 ™* 
- (7_4 — X! fe+70 ' (U -Tio)l[t 4 ,+oo[(«o) -7' (*i -«)] (*i -w)l[ti,+oo[(w) 

i=l 

1 ™* 
~rX] [ Cr 0^ 4 ( ii - M o)l[t 1 .+oo[("o) -CT _4 (t l -u)l[ ti:+00 [(u)] & 
i=l 

1 "* 

o" -4 — X! t 7 ° ' (^"^Ifc.+ooKwo) -7- (*t -w)] (*» ~'")l[t I ,+oo[(w) 



rr 

i=i 



n 

i=l 



= 0(1) +0(1)^0. 

and finally that a.s. 

1 ™* 

Cr : nWl2 = ^cT 2 — X! & "" 7 ° ' (*« ~ U °)] 1 [t«,+oo[('"o) 
i=l 

1 "* 
- cr_2 — X! [&+ 7 ° • (*i - «o)l[ti,+oo[(uo) -27- (tj-u)] l[ 4i .+oo[(w) 
i=l 

1 "* 
= I^X^' K^^.+oclK) -cr _2 l[t„ +00 [(w)] 
i=l 

1 ™* 

+ ~X [ _a o~ 2 7o • (*» -wo)l[ti,+oo[(wo) -^(70 ■ (^-uo)l[t,,+oo[(wo) 



i=l 



-27.(t«-«)l [tli+oo[ («))] 

o(l) + o(l)^0, 
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Proposition 7.11. Let < S, and let (p n )nen be a positive sequence such that, as n — > +00 

Pn = 0(1) (7.20) 

n-3(logn)- jOr ; 1 ^0 (7.21) 
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and denote 

B c (9 0l 6 Pn ) = {9eG, \\6-6o\\ ^S Pn }, 
Then, under Assumptions (A1)-(A4), a.s., there exists e > such that, for any n large enough 

sup _L[Z 1: „(Xi :n |0) - h:n{X 1:n \e n )} < -e. (7.22) 

0GB c (0 o ,<5pr>) n Pn 

Sup -\{h;n(X 1:n \e) - Iv.niXv.nleo)] ^ ~e. (7.23) 

eeBc(6 ,5 Prl ) n Pn 

Proof of Proposition 7.11. This proposition is to be compared to the regularity condition imposed 
in Ghosh et al. (2006) (see their condition (A4) in Chapter 4). The aim of this proposition is to 
show that our model satisfies to a somewhat stronger version of that condition. 

Let < 5. Notice first that, similarly to what was done in (6.20), we are able to deduce that 
a.s. 

-Pii»(*i:»|0) - h:n(X 1:n \9 n )] < -[h :n (X lm \e) - h: n {X 1:n \9 )} =: i n (9). (7.24) 

n n 

where i n is defined over R x [u, u] x R+ D 9 D B c (9 a , 5p n ) by 

2 1 n 1 ™ 

i n (fi) = log -| + 1 + —5 ]T(£ 2 - a 2 ) j Vfe + KVO, U) - Ufa U)] 2 . (7.25) 

i=l i=\ 

2 2 -i n 1 " 

= log^ + l-^ + — ^|- CTo 2 )-— ^{[^+ M (7 ? o,ii)-M(7 ? ,i,)] 2 -cr 2 }. (7.26) 

i=\ 4=1 

From (7.24) it is clear that we need only prove (7.23) to end the proof. 

The rest of this proof is divided into 6 major steps. Step 1 shows that for a given n the 
suprcmum considered is reached on a point 9 n . Step 2 and 3 focus on obtaining useful majorations 
of the supremum. Step 4 is dedicated to proving that the sequence 9 n admits an accumulation 
point (the coordinates of which satisfy to some conditions) , while step 5 makes use of this last fact 
to effectively dominate the suprcmum. Step 6 wraps up the proof. 

Step 1. We first show that a.s. for any n there exists 9 n € Kx[«, u] x R+ such that \\9 n — 0q\\ > 5p n 
and 

i n {8 n )= sup i n {9). (7.27) 

eeB<=(e ,s Pn ) 

Let n€N and let (#n,fc)fceN be a sequence of points in B c (9q, Sp n ) such that 

lim i n {9 n ^ k ) = sup i n {9). 
k—>+oo &eB"(e ,s Pn ) 

From (7.25) it is obvious that er 2 k is bounded: if it was not, we would be able to extract a 
subsequence such that er 2 fe . would go to +oo and thus i n (9n,k) would go to — oo. For the very 
same reason, ^ n ,k too is bounded. Recalling that u n _k is bounded too by definition, we now see 
that there exists a subsequence (0n,fe,)jeN in B c (9o,6p n ) and a point 9 n in B c (9o,Sp n ) (i.e. in 
R x [u, u] x R + . and such that \\9 n — 0o\\ ^ 5p n ) such that (6 n /c,)jeN > 9 n . 

j— > + oo 

Finally from (7.25) again it is easy to see that a 2 > for if it was not i n (9 n ,k ) would go to 
—00 once again, unless (by continuity of p with regard to 77) & + p(r)o,ti) — p,{r) n ,ti) = for all 
i ^ n which a.s. does not happen. 

Step 2. From the previous step and the continuity of i„ with regard to 9 we are able to write 

SUp -{h:n(X 1:n \9) - ll:n(X 1:n \6o)] = i n (9 n ). (7.28) 

eeB"(e ,8 Pri ) n 
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where {O n )n<=N is the sequence defined in Step 1. We now derive a convenient majoration o£i n (9 n ). 
Expanding from (7.26) we get 

/ 2 2\i n 1™ 

i n {6 n )= logff + l-fa +— £(£-0*)- X]{Ki + M%-t i )-M('7,ti)] 2 -^} 

V a n a nJ n<7 i= i n<J i=X 

f 2 2 \ 2 2 n 1 n 

log fa + 1 - f| ) + f^ £(C 2 - ol) -ij $>fo„ t.) - M (^, i,)] 2 

2 ™ 
2 X][M( J 7o>*i)-/' t (»7n.*i)]& 

Ti.fr ^ *■ — • 



" i=l 



Thanks to Lemma 7.8, we know that there exists C\ £ M+ such that 

/ 2 2 \ 2 2 n 

1 2 - 

2 C i\\Vn - Vo\\ 2 j5]WlO,ti)-(i(t|n,t# 



ncr n ■ 



From there, the Law of the Iterated Logarithm and a factorisation of the last term together with 
Corollary 7.7 lead to: 

in(On) < flog °-\ + 1 " °-\ )+ ^Wl ~ 4\Rl.n ~ idll^ - Vof 



— 2 X]^ '^) - ^"'*^] 2 ^ 2 >' 



« \ ,— 



where a.s. i?i.„ = O In 2(loglogn)2 j and i?2.n = O(logn). Lemma 7.3 ensures there exists 
C 2 £ K+ such that 

i n (6 n ) < (log -| + l - -| ) + J-|(j r 2 - CT g|J?!. ri - -^-CiHr/n - 770II 2 + -^Gjn* ||r?„ - ?7o||#2,« 

We thus deduce that there exists C £ R+ such that: 

*»(*») ^ flog^l + 1 - ff ") _ -lch„ - r/oll 2 + -i-||0„ - o ||fln (7.29) 

V (7 (J / (7 (7 

\ v n ^ n / w n w n 

where a.s. R n = O I n~i logn) . Notice in particular that, due to (7.21), R n = o(p n ). 

Step 3. We obtain two majorations, (7.31) and (7.32), that we will make use of in the coming 
steps. Using a conversion of 8 — (7, u, a 2 ) into the spherical coordinate system we write 8 n as 

O n = (r„ cos ip n cos <j) n , r„ sin ip n cos n , r„ sin <j> n ) , 

where 

(r„,Vn,0„) £«;x [0, 2tt]x]0, tt[, 

and deduce from (7.29) that 

*„(*„) < flog -^— + 1 - -4—) Cr n °-^ + -±-R n (7.30) 

\ r„ sin n r„ sin 0„ / sin <p n sin n 

< flog J ° + 1 fa—) + -^— [i?„ - Cr n cos 2 0„] . (7.31) 

V r n sm<p n r n sin (p n J sm0 n 
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From (7.30) we also get the following majoration 

in(0n) < flog g ° , + 1 S— ) + -r^—Rn. (7.32) 

\ r„ sin n r„ sin <p„ J sm 0„ 

Step 4. We show that the sequence (#„)„ 6 n we built, converges to a finite limit 9^ (extracting a 
subsequence if necessary). Extracting a subsequence if necessary, we can assume that (ip n , </>„) — > 
( , >poo,<t > oo) € [0, 27r] x [0, 7r]. We consider the two following mutually exclusive situations. 
Situation A: (j)^ = mod n. In this situation, there exists e > such that for any n large 
enough, 



[Rn - Cr n cos 2 <p n ] = f Ccos 2 (/>„ j r,. 



< -er n , 

because a.s. R n = o(r„) (since R n = o(p„) and r„ < p„). Used together with (7.31), this leads to 



„ am <^ n 



for any n large enough and hence i n (0 n ) ~* ~ o° whether r n goes to zero or not. 

Situation B: tfioo ^ mod n. In this situation, from (7.32), we see that r„ — ► and r„ — > +oo 

both lead to i n (9 n ) — > — oo. 

Observing that i„(0) converges a.s. to a finite value for any 9 £ as n — > +oo, we see 
that lim n — j-oo i„(0„) = — oo is not possible by construction of the sequence #„, and deduce that, 
extracting a subsequence if necessary, there exists 

(roo.^oo.&o) e K; x [0, 2tt]x]0, tt[, 

such that lim n — > +00 #„ = #00. Notice that in particular, er 2 ^ > 0. 

Step 5. We will now end the proof by showing that there exists e > such that for any n large 
enough 

in{0 n ) ^ -ep 2 n . (7.33) 

We consider the two following mutually exclusive situations. 
Situation A: a^ 7^ cr 2 . In this situation, from (7.29) we get 

/ 2 2 \ 1 

i n (8 n ) < ( log^ + 1 - -§■ ) + — \\0 n - 9 \\ Rn 

and the right-hand side converges to 

io g 4+i-4)<°- 

There hence exists e > such that for any n large enough 

in(0 n ) ^ -e. 

Since p n = 0(1) by (7.20), (7.33) is a direct consequence of this. 
Situation B: ct 2 ^ = cr 2 . In this situation, recalling that for any x > 

(x-1) 2 (x-1) 3 
logs + 1 - X ^ -- — + — , 
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we deduce from (7.29) that for any n large enough 



^K-KI-i 



1 (a, 



o -7- 1 ^-Cllr?^ — Tyolr "f — 2 11^- — ' 



<r-±-i 



1 a, 



'D 



<4f4-i 



<; 



4 V cr, 



2 



«"n 



5"C||?7 n -77 || 2 + — 

(7 0~ 



'oil Rra 



— C||r?„ 



?7o|| + ^r 



Rn 



-2 {-c [K 2 - a 2 ) 2 - \\ Vn - ryoll 2 ] + \\e n e \\Rn} 



- 8q\\ Rn 



where c = min(l/4, C) > 0. It follows that for any n large enough 



i n {0 n ) < -o (-c||^ n - 



?„ — ^oll-RnJ 



<^||^-^o||(i?„-c||^-0o||). 



Thus, for any n large enough 



o< 



i lie. 



Pn 



7,i - PQ I 
Pn 



Recalling that 



0o|| ^ <5pn, 

-Rn = 0(p„), 
CT 2 ->• CT 2 > 



we obtain for any n large enough, 

i n (e n ) < 4^ 



"Cg J ^ < 



C«5 2 2 



o5 2 
3^ 



Pn- 



Hence (7.33) holds in this situation too: it suffices to take e 

We just proved that (7.33) holds in both cases considered. 
Step 6. (7.23) is a consequence of (7.28) and (7.33). 



cP 
3a 2 



□ 



Lemma 7.12. Let < 5 < 1 then under Assumptions (Al)-(A^) and conditions (2.6), a.s. there 
exists a constant C € i?i such that for any n large enough and for any 1 ^ ii, 12,^3 ^ 3 



i dH* 1:n (x Un \e) 



n d^di^di. 



^C 



(7.34) 



for any 9 <G B(9q, Sd n ). 



Proof of Lemma 7.12. Let < S < 1. We will prove (7.34) stands true for any 1 ^ ii,i2,t3 ^ 3. 
First notice that for n large enough, 9 H> l\. n {X\-. n \0) is indeed infinitely continuously diffcrentiable 
over B(9o,6d n ) by definition of the pseudo-problem. Any 9 subsequently considered within this 
proof is assumed to belong to B(9o,Sd n ). Any convergence subsequently mentioned within this 
proof is uniform in 9 for 9 € B(9$, Sd n ) for any n large enough thanks to Theorem 7.1 and Lemma 
7.6. 
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Proof of (7.34) for /3 = (3,0,0). 



1 dHl. n (X 1:n \6) 

n (<97) 3 



Proof of (7.34) for /3 = (2, 1, 0). 



1 dH* 1:n (X 1:n \9) 



n (dj) 2 du 



1 n 



i=i 



n — )-+oo (J 



(t-u)f(t)dt 



< —\u-u\ 



Proof of (7.34) for /3 = (2,0,1). 



1 d 3 ;* : „(X 1:n |0) 



n (d^) 2 do 2 



1 ™ 



n — ^+oo (J 



(i- U ) 2 /Wdi 



< -^\ u -m 



Proof of (7.34) for (3 = (1,2,0) 

1 dHt :n (x Un \e) 



n dj ■ (du)' 



"t-l^h^+coliu) 



n — ^+oo (J 



7 / f(t)dt 



Proof of (7.34) for /3 = (1, 1, 1). 



1 dH* 1:n (X 1:n \9) 



n djduda 2 



1 ™ 

- V(A 4 - 2 7 • (fc - «))l ]t4 +oo[ (u) 

i=l 
1 "' 

~ X! [& + 7o ' ( il ~~ ""o) 1 ]*.. +oo[("o) - 27 • (ii - u)] l] tli +00 [(u) 



n — >- + oo (j 4 



min(u.uo) 



lo-(t-u )f(t)dt-2 y(t-u)f(t)dt 



And this limit is bounded by -pc\u — u\(\j\ + |7o|)- 
Proof of (7.34) for /3 = (1,0,2). 



I dH* Un (X Un \6) 



n dj ■ (dcr 2 y 



1 ™ 

-^2 [Xi-J- (U -u)l] tit+00 [(u)] (U -u)l] tii+00 [(u) 

i=\ 
1 ™* 

~5Z [^ +7 ° ' (** _u o)l]t !: +oo[("o) -7- {U -u)] {U -U)l] ti>+00 [(u) 



n — ^+oo <jV 



i(u,uq) 



7o • (* - uo)(* ~ u)/(t) dt - / 7 • (* - «) 2 /W dt 



And this limit is bounded by -%|m — m| 2 (ItI + | To I ) ■ 
Proof of (7.34) for /3 = (0,3,0). 

1 &l* 1:n (X 1:n \0) 
n (9m) 3 
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Proof of (7.34) for /3 = (0, 2, 1). 



1 dnt n (X 1:n \8) 



n (du) 2 da 2 



■2_,hu,+oc[(u) 



^—aT 



n — ^ + oo O" 



f(t)at 



< 



Proof of (7.34) for (3 = (0, 1, 2). 



1 dHt. n (X 1:n \6) 



n du(da 2 y 



n — ^+oo (j® 



l ™ 

- 2J [Zi - 7- (*i - u )l]ti,+oo[(u)] 7l]t(,+oo[(«) 



1 " 

- 5Z [& + To • (*» - wo)l]t(, +oo[("o) - 7 • (*i - u)] jl] U , +«,[(«) 



770 • (t - u )f(t) dt - / 7 2 (i ~ «)/(«) dt 



And this limit is bounded by -§z\u — u|(|7 2 | + ItotD- 
Proof of (7.34) for /3 = (0,0,3). 



1 d 3 l* 1:n (X Un \6) 



n (dcr 2 ) 



2\:5 



3 1 



-1 + — ~^2 [ x i~7- (*i - w)l]*i,+oo[(w)]' 



o n 



3 1 



-i + — -^2 [&+">" (*» -■"o)i]t»,+oo[( , "o) -7- (*i -M)i ]t ^ +oc ,[(u)]' 



n — ^+oo (j 



-cr 2 + 3(a 2 + / ° 7o 2 (t- Uo ) 2 /(t)di 



fmiii(ii,iio) 

77o • (t - uo)(* - «)/(*) di + / 7 2 (i - u) 2 /(t) dt 



And this limit is bounded by -^ [3a 2 + a 2 + (\j\ + |7o|) 2 (u - u) 2 ] . 

(7.34) is thus a direct consequence of both the uniform convergences mentioned above and the 
trivial majoration of all the limits involved by a fixed constant C for any n large enough. □ 
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